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REMARKS 

In response to the final Office Action dated February 21 , 2003, claims 21-92 have 
been canceled and new claims 93 and 94 have been added. Therefore, claims 1-20, 93 
and 94 are now in the case. Reexamination and reconsideration of the amended 
application are requested. 

Rebuttal to "Response to Arguments" 
In the "Response to Arguments" section, the Examiner disagreed with the 
Applicants 1 aigument that Sambonsugi et al. do not disclose a prediction module that 
provides predictions for a value of each of a plurality of pixels. In particular, the Examiner 
stated that Sambonsugi et al. "shows this feature on FIG. 13 and column 20, lines 52-67." 

In response to the Examiner's argument, the Applicants respectfully submit the 
following rebuttal. Independent claims 1,11 and 16 all claim the material feature of 
providing predictions for a value of each pixel in an image sequence. The Applicants 
respectfully invite the Examiner to note at least two points. First, predictions are made 
of a pixel value . Thus, what are claimed are predictions of a pixel value , not a shape . 
Moreover, since predictions are being made, this means that the pixel value is 
unknown . Second, predictions are in the plural . In other words, more than one 
prediction is provided for each pixel value , Thus, the Applicants' claimed invention 
includes plural pixel value predictions . 

In contrast, Sambonsugi et al. merely disclose a single , shape prediction for each 
block into which an image was divided. Furthermore, the shape data discussed in 
Sambonsugi et al. Is comprised of pixel values. Each pixel in the shape data has a 
value of either "255" or "0". Each pixel value is known . There can be no prediction of 
pixel values in the shape data because there is no predicting to be done. The pixel 
values have already been determined » 
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In particular, Sambonsugl et al. teaches a prediction section 'Tor predicting a 
position or shape of the object on the current frame from a frame from which an object 
region has already been extracted" (col. 4, lines 47-49). This prediction section aids in 
the extraction of the object from a frame (col. 4, lines 52-54). In addition, the apparatus 
of Sambonsugi et al. includes object extraction sections that perform a function similar 
to the prediction section (col. 5, lines 3-11). 

Figure 13 and the section of Sambonsugi et al. cited by the Examiner discuss 
this shape prediction and extraction in more detail. Specifically, shape prediction "is 
performed by block matching " (col. 20, lines 52-53; emphasis added). As shown in FIG. 
13 of Sambonsugi et al. f block matching is carried out by segmenting an image inside 
the current frame into equal-sized blocks (col. 20, lines 53-54). Each block from the 
current frame is matched to a block in the reference frame. The blocks are matched 
using texture (col. 20, lines 55-58). 

The reference frame contains shape data that represents an object region and 
has already been created (col. 20, lines 58-59). In other words, the shape data Is 
known prior to the performing of shape prediction. Shape data is obtained by defining 
pixel values. Specifically, if a pixel belongs to the object region (i.e., is on the object), 
then the value of that pixel is "255". Otherwise, for pixels that lie outside of the object, 
the pixel value is "0" (col. 20, lines 59-62). In other words, the shape data contains pixel 
values of either "255" or "0 n . Once again, the pixel values are known before any shape 
prediction is performed. Shape prediction cannot be achieved in Sambonsugi et al. 
without knowing each pixel value in the shape data. 

Once the pixel values of the shape data are known, the shape data 
corresponding to the block within the reference frame is pasted to the corresponding 
block in the current frame (col. 20, lines 62-64). This is done for all the blocks of the 
current frame so that the image in the current frame is filled with shape data (col. 20, 
lines 64-67 to col. 21 , line 1 ). Thus, the image in the current frame is filled with pixel 
values of either "255" or "0". This allows the system of Sambonsugi et al. to 
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discriminate the image from the background (col. 21, lines 1-2). In this manner, an 
image (or object) can be extracted from the current frame (col. 21 , lines 2-3). 

As described above, the system and method disclosed in Sambonsugi et al. 
merely teach shape prediction to extract an object from a current frame using a 
reference frame. Texture is used to matches blocks between the current frame and the 
reference frame. Shape data from the reference frame then is pasted from the 
reference frame block into the current frame block. 



This shape data contains pixel having one of two values: "255" or "0". These 
pixel values are determined before shape prediction occurs. Nowhere do Sambonsugi 
et al. teach making prediction for each pixel value in the shape data. In fact, there is no 
need for prediction , since the value of each pixel in the shape data is already known . In 
particular, the " shape data representing an object region on this reference frame has 
already been created '' (col. 20, lines 58-59; emphasis added). In other words, the 
value of each pixel (i.e. shape data) is already determined . There is no need to make 
even a single prediction about pixel value (let alone more than one pixel value 
prediction, as in the Applicants' claimed invention) when each pixel value is already 
known. 



Based on the arguments above and below, the Applicants respectfully maintain 
that their claimed feature of providing predictions for a value of each pixel in an image 
sequence is not taught or disclosed by Sambonsugi et al.. Accordingly, the Applicants 
respectfully request the Examiner to reconsider his response to the Applicants' 
arguments and pass this application to issue. 

Section 102(e) Rejections 
The Office Action rejected claims 1-7 and 11-19 under 35 U.S.C. § 102(e) as being 
anticipated by Sambonsugi et al. (U.S. Patent No. 6,335,985). The Office Action stated 
that Sambonsugi et al. disclose all the elements of the Applicants' claimed invention. 
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In response, the Applicants respectfully traverse these rejections based on the 
following legal and technical analysis. In general, the Applicants submit that 
Sambonsugi et al. lack at least one feature of the Applicants' claimed invention. In 
particular, Sambonsugi et al. do not disclose, either explicitly or implicitly, the material 
claimed feature of providing predictions for a value of each pixel in an image sequence. 

Independent Claim 1 

Independent claim 1 of the Applicants' claimed invention includes a system for 
maintaining a background model of an image sequence having a plurality of pixels. The 
system further includes a pixel processing module that processes the image sequence on 
a pixel scale. In addition, the system includes a prediction module that provides 
predictions for a value of each of the plurality of pixels . The system also includes at least 
one refinement module that processes the image sequence on a spatial scale other than 
the pixel scale. 

The Applicants' specification includes an example implementation of the claimed 
material feature of providing predictions for a value of each of the pixels. Specifically, the 
Applicants' specification sets forth a working example whereby there are two predictions of 
pixel value are made for each pixel. One of the predictions is based on actual history and 
the other prediction is based on the predicted history of a pixel (page 28, lines 3-5). If the 
actual value of pixel differs from either one of its two predicted values by more than a 
certain amount, then that pixel is declared a foreground pixel (page 28, lines 2-3). It 
should be noted that predictions are made of pixel values because the pixel values are 
unknown . 

In striking contrast, Sambonsugi et al. t merely teach a system and method that 
uses a single prediction for each block to predict shape . Any pixel values taught in 
Sambonsugi et al. are contained in the shape data. Each pixel value is known . There 
can be no prediction of pixel values in the shape data because there is no predicting to 
be done. The pixel values have already been determined . 
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While Sambonsugi et al. do discuss prediction, they merely disclose an object 
extraction apparatus that performs shape prediction of an object within a frame. In 
particular, that apparatus includes a prediction section "for predicting a position or shape of 
the object on the current frame from a frame from which an object region has already been 
extracted" (col. 4, lines 47-49). This prediction section aids in the extraction of the object 
from a frame (col. 4, lines 52-54). In addition, the apparatus of Sambonsugi et al. includes 
object extraction sections that perform a function similar to the prediction section (col. 5, 
lines 3-11). Unlike the Applicants' claimed prediction module t hat provides pixel value 
predictions , however, the prediction section and object extraction sections of Sambonsugi 
et al. merely discuss shape prediction . Moreover any pixel values used in Sambonsugi et 
al. are known pixel values . Thus, there can be no pixel value prediction in Sambonsugi et 
al.. 

The Applicants, therefore, respectfully traverse this rejection of independent 
claim 1 because Sambonsugi et al. do not disclose, either explicitly or implicitly, the 
material claimed feature of providing predictions for a value of each pixel in an image 
sequence. Because of this missing feature, the §102 rejection cannot stand. 

Independent Claim 1 1 

Independent claim 1 1 includes a computer-readable medium having computer- 
executable modules. These modules include a pixel processing module that processes an 
image sequence on a pixel scale. The pixel processing module further includes a 
prediction module that calculates predictions for a value of each pixel within the image 
sequence. In addition, the modules include at least one refinement module that processes 
the image sequence on a spatial scale other than the pixel scale. In contrast, as noted 
above, Sambonsugi et al. merely teach a system and method that uses a single prediction 
for each block to predict shape . Nowhere are the Applicants 1 claimed pixel value 
predictions disclosed. 

Independent Claim 16 
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Independent claim 16 includes a method for maintaining a background model of an 
image sequence having a plurality of pixels. The method includes processing the image 
sequence on a pixel scale so as to determine a current background model and provide an 
initial assignment for each of the plurality of pixels. The method also includes calculating 
predictors for a value of each of the plurality of pixels, and refining the pixel processing by 
processing on a spatial scale other than the pixel scale to further refine at least one of: (a) 
the current background model; (b) the initial pixel assignments. On the other hand, as 
discussed above, Sambonsugi et al. merely teach a system and method that performs a 
single prediction for each block to predict shape . This is vastly different from the 
Applicants 1 claimed invention including predictions of a pixel value. 

Because the Applicants' claimed invention includes features neither taught, 
disclosed nor suggested by Sambonsugi et al., the Applicants respectfully submit that 
the rejections of independent claims 1,11 and 16 under 35 U.S.C. § 102(e) as being 
anticipated by Sambonsugi et al. has been overcome based on the arguments set forth 
above and below. Moreover, rejected claims 2-7 depend from independent claim 1 , 
rejected claims 12-15 depend from independent claim 11, and rejected claims 17-19 
depend from independent claim 16 and are therefore also novel over Sambonsugi et al. 
(MPEP § 2143.03). The Applicants, therefore, respectfully request reexamination, 
reconsideration and withdrawal of the rejection of claims 1-7 and 11-19 under 35 U.S,C. 
§ 102(e) as being anticipated by Sambonsugi et al. based on the arguments above and 
below. 



The Office Action rejected claims 8 and 20 under 35 U.S.C. § 103(a) as being 
unpatentable over Sambonsugi et al. in view of Jain et al. (U.S. Patent No. 6,263,091 ). 
The Office Action contended that Sambonsugi et al. disclose all elements of the 
Applicants' claimed invention except for disclosing speckle removal. However, the Office 
Action stated that Jain et al. disclose a technique to isolate foreground and background 
using speckle removal. Therefore, the Office Action asserted that rt would have been 
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obvious to use speckle removal as an enhancement technique because speckle removal 
is well known in the art to aid in the removal of noise, dirt, breaks and smudges in input 
images. 

In response, the Applicants respectfully traverse these rejections based on the 
following legal and technical analysis. The Applicants submit that Sambonsugi et al. 
and Jain et al. are lacking at least element of the Applicants' claimed invention. In 
particular, Sambonsugi et al. and Jain et al. do not disclose, either explicitly or implicitly, 
the material claimed feature of providing predictions for a value of each pixel in an 
image sequence. Further, Sambonsugi et al. and Jain et al. fail to appreciate the 
advantages of this claimed feature. In addition, there is no technical suggestion or 
motivation disclosed in Sambonsugi et al. or Jain et al. to define this claimed feature. 
Thus, the Applicants' submit that Sambonsugi et al. and Jain et al. cannot make obvious 
the Applicants' claimed feature of providing predictions for a value of each pixel in an 
image sequence. 

To make a prima facie showing of obviousness, all of the claimed features of an 
Applicants' invention must be considered, especially when thev are missing from the prior 
art . If a claimed feature is not disclosed in the prior art and has advantages not 
appreciated bv the prior art , then no prima facie showing of obviousness has been made. 
The Federal Circuit Court has held that it was an error not to distinguish claims over a 
combination of prior art references where a material limitation in the claimed system and 
its purpose was not taught therein. In re Fine, 837 F.2d 1 071 , 5 USPQ2d 1 596 (Fed. Cir. 
1988). Moreover, as stated in the MPEP, if a prior art reference does not disclose, 
suggest or provide any motivation for at least one claimed feature of an Applicants' 
invention, then a prima facie case of obviousness has not been established (MPEP § 
2142). 

Independent Claims 1 and 16 and Dependent Claims 

As discussed above, independent claim 1 of the Applicants' claimed inventbn 
includes a system for maintaining a background model of an image sequence having a 
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plurality of pixels. The system further includes a pixel processing module that processes 
the image sequence on a pixel scale and a prediction module that provides pixel value 
predictions for each of the plurality of pixels. In addition, the system includes at least one 
refinement module that processes the image sequence on a spatial scale other than the 
pixel scale. In contrast, as noted above, Sambonsugi et aL merely disclose an object 
extraction apparatus that performs shape prediction of an object within a frame. 

As also discussed above, independent claim 16 includes a method for maintaining 
a background model of an image sequence having a plurality of pixels that includes 
calculating predictors for a value of each of the plurality of pixels . Once again, 
Sambonsugi et al. merely teaches disclose shape prediction that is nothing like the 
Applicants 1 claimed invention. 

In addition, Sambonsugi et al. fall to provide any motivation, suggestion or 
desirability to modify their object extraction apparatus to include a prediction module 
that provides predictions for a value of each pixel in an image sequence. One reason 
for this is that the technique used in Sambonsugi et al. is for performing shape 
prediction of an object within a frame. This shape prediction uses shape data, which 
includes pixel values that are already known . There can be no motivation to predict 
something that is already known. Thus, absent any type of teaching, motivation or 
suggestion Sambonsugi et al. cannot render the Applicants' invention obvious (MPEP § 
2143.01). 

Jain et al. add nothing to the cited combination that would render the Applicants' . 
claimed invention obvious. Jain et al. merely disclose a system and a method for 
segmenting foreground and background portions of digitized images. The Applicants' 
claimed feature of a prediction module that provides predictions for a value of each pixel 
in an Image sequence is not discussed. Consequently, no motivation or suggestion for 
this claimed feature of the Applicants 1 invention is provided. Absent this teaching, 
motivation or suggestion, Jain et al. cannot render the Applicants' claimed invention 
obvious (MPEP § 2143.01). 
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Sambonsugi et al. and Jain et al. also both fail to appreciate or recognize the 
advantages of the Applicants' claimed feature of a prediction module that provides 
predictions for a value of each pixel in an image sequence. More specifically, the use of 
multiple predictions "provides the pixel processing module 310 with the ability to 
accurately maintain a model of the background even if the background is briefly 
concealed by a foreground objecf (page 28, lines 11-13). For example, if an actual 
pixel history and a predicted pixel history are used as two predictors for a value of each 
pixel, if one predictor becomes corrupted the other predictor will continue to predict the 
background (page 28, lines 13-1 5). Neither Sambonsugi et al. nor Jain et al. discuss or 
appreciate these advantages of the Applicants' claimed feature of a prediction module 
providing pixel value predictions. 

The Applicants, therefore, submit that obviousness cannot be established since 
neither Sambonsugi et al. nor Jain et al. teach, disclose, suggest or provide any 
motivation for the Applicants' claimed feature of a prediction module having pixel value 
predictions. In addition to explicitly lacking this feature, Sambonsugi et al. and Jain et 
al. also fail to implicitly disclose, suggest, or provide motivation for this feature. In 
particular, Sambonsugi et al. and Jain et al. lack any suggestion and fail to provide any 
motivation for Applicants' claimed feature. Further, both Sambonsugi et al. and Jain et al. 
fail to appreciate advantages of this claimed feature. 

Therefore, as set forth in In re Fine and MPEP § 2142, Sambonsugi et al. and 
Jain et al M either alone or in combination, do not render the Applicants' claimed 
invention obvious because the references are missing at least one material feature of 
the Applicants' claimed invention. Consequently, because a prima facie case of 
obviousness cannot be established due to the lack of "some teaching, suggestion, or 
incentive supporting the combination", the rejection must be withdrawn. ACS Hospital 
Systems. Inc. v. Montefiore Hospital . 732 F.2d 1572, 1577, 221 USPQ 929, 933 (Fed. 
Cir. 1984); MPEP 2143.01. 
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Accordingly, the Applicants respectfully submit that independent claims 1 and 16 
are patentable under 35 U.S.C. § 103(a) over Sambonsugl et al. in view of Jain et al. 
based the legal and technical arguments set forth above and below. Moreover, claim 8 
depends from independent claim 1 and claim 20 depends from independent claim 16, and 
are also nonobvious over Sambonsugl et al. in view of Jain et al. (MPEP § 2143.03). The 
Applicants, therefore, respectfully request reexamination, reconsideration and withdrawal 
of the rejection of claims 8 and 20. 



The Office Action rejected claims 9 and 10 under 35 U.S.C. § 103(a) as being 
unpatentable over Sambonsugi et al.. The Office Action contended that Sambonsugi et al. 
disclose all elements of the Applicants' claimed invention including a postprocessing 
module. Therefore, the Office Action maintained that it would have been obvious for one 
of ordinary skill in the art to come up with a method wherein the postprocessing module 
provides enhancement after the pixel processing module and before the frame processing 
module in order to output a better quality sequence of images. 

In response, the Applicants respectfully traverse these rejections based on the 
following legal and technical analysis. In particular, Sambonsugi et al. do not disclose, 
either explicitly or implicitly, the material claimed feature of providing predictions for a 
value of each pixel in an image sequence. Further, Sambonsugi et al. fail to appreciate 
the advantages of this claimed feature. In addition, there is no technical suggestion or 
motivation disclosed in Sambonsugi et al. to define this claimed feature. Thus, the 
Applicants' submit that Sambonsugi et al. cannot make obvious the Applicants' claimed 
feature of providing predictions for a value of each Pixel In an image sequence. 

Independent Claim 1 and Dependent Claims 
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As discussed above, independent claim 1 of the Applicants' claimed invention 



includes a system for maintaining a background model of an image sequence having a 
plurality of pixels. The system further includes a pixel processing module that processes 
the image sequence on a pixel scale and a prediction module that provides predictions for 
a value of each of the plurality of pixels. In addition, the system includes at least one 
refinement module that processes the image sequence on a spatial scale other than the 
pixel scale. In contrast, Sambonsugi et al. disclose an object extraction apparatus that 
performs shape prediction. 

In addition, Sambonsugi et al. fail to provide any motivation, suggestion or 
desirability to modify their object extraction apparatus to include a prediction module 
having pixel value predictions. One reason for this is that the technique used in 
Sambonsugi et al. is for performing shape prediction of an object within a frame. This 
shape prediction uses shape data, which includes pixel values that are already known . 
There can be no motivation to predict something that is already known. Thus, absent 
any type of teaching, motivation or suggestion Sambonsugi et al. cannot render the 
Applicants 1 invention obvious (MPEP § 2143.01). 

In addition, Sambonsugi et al. fail to appreciate or recognize the advantages of 
the Applicants' claimed feature of a prediction module having pixel value predictions. 
More specifically, the use of multiple predictors "provides the pixel processing module 
310 with the ability to accurately maintain a model of the background even if the 
background is briefly concealed by a foreground object" (page 28, lines 1 1-13). For 
example, if an actual pixel history and a predicted pixel history are used as two 
predictors for a value of each pixel, if one predictor becomes corrupted the other 
predictor will continue to predict the background (page 28, lines 1 3-1 5). Sambonsugi et 
al. do not discuss or appreciate these advantages of this claimed feature of the 
Applicants' claimed invention. 

The Applicants, therefore, submit that obviousness cannot be established since 
Sambonsugi et al. do not teach, disclose, suggest or provide any motivation for the 

16 of 18 

Received from < 8092788054 > at 9/16/03 8:01:11 PM [Eastern Daylight Time] 



9 

785064 



05/1B/2003 16:05 805278BBB4 LYON & HARR| i i u i n l PAGE 21 

Serial Na: 09/400,346 Attomdy Docket No: MCS-058-09 

Applicants' claimed feature of a prediction module having pixel value predictions. In 
addition to explicitly lacking this feature, Sambonsugi et al. also fail to implicitly disclose, 
suggest, or provide motivation for this feature. In particular, Sambonsugi et al. lack any 
suggestion and fail to provide any motivation for Applicants' claimed feature. Further, 
Sambonsugi et al. fail to appreciate advantages of this claimed feature. 



Therefore, as set forth in In re Fine and MPEP § 2142, Sambonsugi et al. do not 
render the Applicants' claimed invention obvious because it is missing at least one 
material feature of the Applicants' claimed invention. Consequently, because a prima 
facie case of obviousness cannot be established due to the lack of "some teaching, 
suggestion, or incentive supporting the combination", the rejection must be withdrawn. 
ACS Hospital Systems. Inc. v. Montefiore Hospital . 732 F.2d 1572, 1577, 221 USPQ 
929, 933 (Fed. Cir. 1984); MPEP 2143.01. 



Accordingly, the Applicants respectfully submit that independent claim 1 is 
patentable under 35 U.S.C. § 103(a) over Sambonsugi et al. based the legal and technical 
arguments set forth above and below. Moreover, claims 9 and 10 depend from 
independent claim 1 and are also nonobvious over Sambonsugi et al. (MPEP § 2143.03). 
The Applicants, therefore, respectfully request reexamination, reconsideration and 
withdrawal of the rejection of claims 9 and 10. 



Conclusion 

In view of the arguments set forth above, the Applicants submit that claims 1-20, 
93 and 94 of the subject application are In immediate condition for allowance. The 
Examiner is respectfully requested to withdraw the outstanding rejections of the claims 
and to pass this application to issue. 

In an effort to expedite and further the prosecution of the subject application, the 
Applicants kindly invite the Examiner to telephone the Applicants 1 attorney at (805) 278- 
8855 if the Examiner has any comments, questions or concerns, wishes to discuss any 
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aspect of the prosecution of this application, or desires any degree of clarification of this 
response. 



Respectfully submitted, 
Dated: May 16, 2003 




CrairS. Fis 

(egistratiofrfJo. 42,535 
Attorney for Applicants 
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